Method and apparatus for photograph finding

ABSTRACT

Digital image data including discrete photographic images of a variety of different subjects, times, and so forth, are collected and analyzed to identify specific features in the photographs. In an embodiment of the invention, distinctive markers are distributed to aid in the identification of particular subject matter. Facial recognition may also be employed. The digital image data is maintained in a database and quarried in response to search requests. The search requests include criteria specifying any feature category or other identifying information, such as date, time, and location that each photograph was taken, associated with each photograph. Candidate images are provided for review by requesters, who may select desired images for purchase or downloading.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.12/874,929, filed Sep. 2, 2010 which is a continuation of U.S.application Ser. No. 12/325,589 filed Dec. 1, 2008, Now U.S. Pat. No.7,844,141, which is a continuation of U.S. application Ser. No.11/056,699 filed Feb. 10, 2005, now U.S. Pat. No. 7,460,737, whichclaims priority pursuant to 35 U.S.C. § 119(e) to U.S. ProvisionalApplication No. 60/544,570, filed Feb. 12, 2004, which applications arespecifically incorporated herein, in their entirety, by reference.

BACKGROUND OF THE INVENTION Field of the Invention

Photography has transformed how people conceive of the world.Photographs allow people to see all sorts of things that are actuallymany miles away and/or years preceding. Photography lets people capturemoments in time and preserve them for years to come.

Often people at a public place notice that a stranger has taken aphotograph of which they would love to have a copy. Alternatively, aftergoing somewhere, a person may bemoan the fact that he did not have aphotograph of the event (in the present context, photograph alsoincludes video, audio, or other representation).

A need exists, therefore, to provide a method and apparatus foridentifying and connecting people with photographs they want. Inaddition, there is a need to provide a method and apparatus forcharacterizing errant photographs stored on computer databases thatmakes use of a variety of attributes to reliably characterizephotographs in such a way as to reduce the amount of manual reviewnecessary to identify and connect people with the photographs they want.

SUMMARY OF THE INVENTION

The present invention provides a method and apparatus that matchespeople with photographs in which they accidentally (or purposely) appearor with photographs of events they have attended.

Specifically, in one embodiment, a web site is created with a databasebackend. The database is seeded with information provided by (1) thephotographer; (2) recovering metadata from the photograph; (3) readingdevices such as a Global Positioning System (GPS) device; (4)referencing the account data of the photographer (i.e., account number,photographer's zip code or area code, etc.); (5) analyzing thephotograph (i.e., computer recognizes eye color, optical characterrecognizes any text found in the photograph, recognizes the number ofpersons, the gender of persons, the hair color, the time of day byoptical character recognizing any clocks in the photograph or analyzingthe lighting conditions, the weather, etc.); (6) photograph qualityinformation; and/or (7) any other information.

A user looking for a photograph would visit the web site and search forcertain criteria. The user is then provided with a gallery of thumbnailsthat match the criteria. When the user identifies a photograph he wantsto own, he can then download the full quality version, or orderprint(s). In a preferred implementation, the user is charged some amountof money that is split between the site owner and the photographer.Alternatively, the user may be charged in some other way, such as byviewing advertisements or by exchanging credits for downloads or by someother payment or a combination thereof.

A more complete understanding of the present invention will be affordedto those skilled in the art, as well as a realization of additionaladvantages and objects thereof, by a consideration of the followingdetailed description of the preferred embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow diagram showing exemplary steps of a method accordingto the invention.

FIG. 2 is a diagram showing an exemplary distinctive marker forphotographic data.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention provides a method and apparatus that matchespeople with photographs in which they accidentally (or purposely) appearor with photographs of events they have attended.

FIG. 1 illustrates exemplary steps of a method 100 according to theinvention. At optional step 102, distinctive markers may be distributedto persons desiring to contribute photographic images to a database. Themarkers may comprise, for example, distinctive bins, badges, or stickersfor placing on objects to be photographed. The markers should bedesigned so as to be easily recognized using automatic recognitionalgorithms, but should not be too conspicuous.

At step 104, image data is collected from a variety of sources. It maybe desirable to accept material from as many sources as possible, toincrease the number of images available for browsing. Optionally, imagesmay be accepted from qualified sources only.

At step 104, source information regarding each photograph is collected.For example, at step 108, the image provider may be asked forinformation, such as the time and date of the photograph, the subjectmatter, location, photographer, etc. Image metadata may also be read toobtain such information. Other ways of associating images to identifyinginformation may include, for example, reading devices such as a GlobalPositioning System (GPS) device attached to a camera or other imagecapture device, or by referencing account data of the image contributor(e.g., account number, photographer's zip code or area code, etc.).

At step 110, image data is analyzed to identify any characteristics thatmay be of interest to users. Such characteristics may include, forexample, eye color, words and sentences, a number or gender of persons,the hair color, time of day, lighting conditions, and so forth. Forfurther example, at step 112, a facial recognition program as known inthe art may be used to analyze any faces appearing in the photos at asufficiently high resolution. At step 114, the images may be analyzedfor the presence of any known markers. And at step 116, other featuresand qualities of the image may be classified, for example, whether it istaken indoors or outdoors, whether it contains people, dogs, cats, orother animals, whether it contains automobiles, airplanes, or otherobjects, and so forth. At step 118, selected feature information andother source information is associated with each image and provided toany suitable relational database.

At step 120, requests specifying search criteria for photographic imagesare received. For example, a user looking for a photograph may visit aweb site hosted by the system and fill out a search form specifyingsearch criteria of interest. The criteria may include specific subjectmatter, times, dates, and locations. For example, “Disneyland ANDMatterhorn AND blue eye AND child AND Jan. 1, 2004 AND morning” wouldsearch for a photograph or photographs taken at Disneyland's Matterhornwith a child who has blue eyes on the morning of Jan. 1, 2004.

At step 122, the image database is queried as known in the art, toidentify images that at least partially match the search criteria. Suchimages may be presented, at step 124, to the user. For example, the usermay be provided with a gallery of “thumbnail” (reduced-size) imagesgenerated from images that match the criteria. When the user identifiesa photograph he wants to own, he can then download the full qualityversion, or order print(s). In a preferred implementation, the user ischarged some amount of money that is split between the site owner andthe photographer. Alternatively, the user may be charged in some otherways such as by viewing advertisements or by exchanging credits fordownloads or by some other payment or a combination thereof. The pricecan be on a sliding scale depending on the quality of the photographthat the user downloads or the size or quality of the print. Forexample, a photograph may cost $1.00 for 1024×768 resolution or $2.00for 1600×1200 resolution. Similarly, a print may cost $1.00 for 3×5 or$5 for 8×10. For downloads, an “upgrade” may be possible by charging thedifference between the resolutions. An automated process may be used toreduce the number of pixels for purposes of having a lower qualityversion to sell.

In addition, a surcharge may be applied (even if no surcharge isrequired) for various enhancements to the photograph, such as“upconverting” to a higher resolution, eliminating red-eye, enhancingshadow, color, or brightness, etc.

Moreover, when a photographer takes photographs, he can be provided withprinted cards bearing a Uniform Resource Locator (URL) and a unique codein order that the user would be able to enter into the web site to findthe photograph or the series of photographs then being taken. Thephotographer can also distribute cards (the printed cards bearing theURL and the unique code or any other cards known to those skilled in theart) to people whom he photographs, whether intentionally orinadvertently. The photographer can further advertise the same (e.g.,the URL and the unique code) via a mark on his camera, a T-shirt, orother means.

Fixed-place cameras can also serve this function (e.g., the ofphotographer). For example, a camera set up at an intersection inHollywood might take and upload one photograph every 10 seconds.

Photographers can also be given accounts and be allowed to uploadphotographs to the site. The database is populated during this process,although additional database information can be added later by web siteusers. In addition, the number of times the photograph has beenpurchased and/or viewed can be a part of the database.

In one embodiment, the method and apparatus of the present inventionshould be capable of face recognition. It should assign values tovarious factors (i.e., ratio of distance between pupils to distance totip of nose, etc.). It would add this information to the database foruploaded photographs. A user can then upload a photograph of the targetperson and the system would then generate the same data from thatphotograph and use it to limit the possible search targets.

A provider of the present method and apparatus or a photographer canalso hand out pins, clothing, or other materials that are marked in away that allows a computer to later recognize them in a photograph. FIG.2 shows an exemplary distinctive marker 200 having an optical code 202,such as a bar code. The marker may have a color combination, distinctiveshape, lettering, bar code, or other optical pattern, or somecombination of the foregoing, that is unique to the marker. The markermay be computer generated, for example, and produced using an end-user'slaser or ink-jet printer. The marker may be associated with specificinformation, for example, a particular user account, photographer,subject matter type, person, event, or location. Users can later searchfor photographs containing an image of the marker.

Numerous distribution mechanisms exist whereby photographs may bedistributed from a source over a wide area network, such as theInternet. In some cases, the photographs are distributed using acentralized server system (such as Napster 2.0, eBay, or from a website). In other cases, the photographs are distributed using adecentralized system (such as Gnutella). In a preferred implementation,the photographs are distributed to a person using the centralized serversystem or using a central hub.

Embodiments of the present invention operate in accordance with at leastone web-hosting mechanism and a plurality of user mechanismscommunicating over a wide area network, such as the Internet.Specifically, a web-hosting mechanism includes a database, an interfaceapplication and a server, wherein the server is adapted to communicatewith a plurality of user mechanisms over a wide area network. It shouldbe appreciated that the mechanisms described can include, but are notlimited to, personal computers, mainframe computers, personal digitalassistances, wireless communication devices and all other physical andwireless connected network devices generally known to those skilled inthe art. It should further be understood that the database depicted caninclude, but is not limited, to RAM, cache memory, flash memory,magnetic disks, optical disks, removable disks, SCSI disks, IDE harddrives, tape drives, and all other types of data storage devices (andcombinations thereof, such as RAID devices) generally known to thoseskilled in the art. In addition, the mechanisms described above are forpurposes of example only and the invention is not limited thereby.

Having thus described several embodiments for photograph finding, itshould be apparent to those skilled in the art that certain advantagesof the system have been achieved. It should also be appreciated thatvarious modifications, adaptations, and alternative embodiments thereofmay be made within the scope and spirit of the present invention. Forexample, in the context of the present invention a photograph caninclude video, audio, and/or other representation of how people conceiveof the world. The invention is defined by the following claims.

1. (canceled)
 2. A method, comprising: a computer system receiving asearch request that includes a digital image; the computer systemanalyzing the digital image to identify characteristics of the digitalimage, including determining a set of one or more colors included in thedigital image; the computer system searching, using the identifiedcharacteristics, information specifying characteristics of respectiveones of a plurality of stored digital images; the computer systemselecting one or more of the plurality of stored digital images, whereinthe selecting is based on a comparison of the identified characteristicsand characteristics specified by the information, wherein the comparedcharacteristics include colors included in the digital images; and thecomputer system returning the selected one or more digital images. 3.The method of claim 2, wherein the set of one or more colors correspondsto colors of one or more predetermined types of objects recognized inthe digital image.
 4. The method of claim 2, wherein the identified andcompared characteristics further include lighting conditions.
 5. Themethod of claim 2, wherein the returning includes causing reduced-sizeversions of the one or more digital images to be displayed via a website.
 6. The method of claim 2, wherein the identified and comparedcharacteristics further include a source of the digital image.
 7. Themethod of claim 6, wherein the source of the digital image is indicatedusing at least one of: a user account of a contributor of the digitalimage or a device used to capture the digital image.
 8. The method ofclaim 2, further comprising: the computer system receiving one or moresearch parameters based on user input; wherein the selecting is furtherbased on one a comparison of the search parameters and parameterscorresponding to respective ones of the plurality of stored digitalimages.
 9. The method of claim 2, wherein the identified and comparedcharacteristics further include types of objects depicted.
 10. A system,comprising: one or more processors; and one or more memories havingprogram instructions stored thereon that are executable by the one ormore processors to cause the system to perform operations comprising:receiving a search request that includes a digital image; analyzing thedigital image to identify characteristics of the digital image,including determining a set of one or more colors included in thedigital image; searching, using the identified characteristics,information specifying characteristics of respective ones of a pluralityof stored digital images; selecting one or more of the plurality ofstored digital images, wherein the selecting is based on a comparison ofthe identified characteristics and characteristics specified by theinformation, wherein the compared characteristics include colorsincluded in the digital images; and returning the selected one or moredigital images.
 11. The system of claim 10, wherein the set of one ormore colors corresponds to colors of one or more predetermined types ofobjects recognized in the digital image.
 12. The system of claim 10,wherein the identified and compared characteristics further includelighting conditions.
 13. The system of claim 10, wherein the identifiedand compared characteristics further include types of objects depicted.14. A non-transitory computer-readable medium having instructions storedthereon that are executable by a computing device to perform operationscomprising: receiving a search request that includes a digital image;analyzing the digital image to identify characteristics of the digitalimage, including determining a set of one or more colors included in thedigital image; searching, using the identified characteristics,information specifying characteristics of respective ones of a pluralityof stored digital images; selecting one or more of the plurality ofstored digital images, wherein the selecting is based on a comparison ofthe identified characteristics and characteristics specified by theinformation, wherein the compared characteristics include colorsincluded in the digital images; and returning the selected one or moredigital images.
 15. The non-transitory computer-readable medium of claim14, wherein the set of one or more colors corresponds to colors of oneor more predetermined types of objects recognized in the digital image.16. The non-transitory computer-readable medium of claim 14, wherein theidentified and compared characteristics further include lightingconditions.
 17. The non-transitory computer-readable medium of claim 14,wherein the identified and compared characteristics further include asource of the digital image.
 18. The non-transitory computer-readablemedium of claim 17, wherein the source of the digital image is indicatedusing at least one of: a user account of a contributor of the digitalimage or a device used to capture the digital image.
 19. Thenon-transitory computer-readable medium of claim 14, wherein theoperations further comprise: receiving one or more search parametersbased on user input; wherein the selecting is further based on one acomparison of the search parameters and parameters corresponding torespective ones of the plurality of stored digital images.
 20. Thenon-transitory computer-readable medium of claim 14, wherein theidentified and compared characteristics further include types of objectsdepicted.